Snap and Northeastern University Researchers Propose EfficientFormer: A Vision Transformer That Runs As Fast As MobileNet While Maintaining High Performance In natural language processing, the Transformer is a unique design that seeks to solve sequence-to-sequence tasks while also resolving long-range dependencies. Vision Transformers (ViT) have demonstrated excellent results on computer vision benchmarks in recent years. On the other hand, they are usually times slower than lightweight convolutional networks because of the large number of parameters and model architecture, such as the attention mechanism. As a result, deploying ViT for real-time applications is difficult, especially on hardware with limited resources, such as mobile devices. Snap Inc. and Northeastern University collaborated on a new study that answers this fundamental question and suggests a new ViT paradigm.

Thank you so much for signing up for my AI Newsletter. In the last few days, we were doing deep research in AI-related research updates and we were able to find these cool ones. But before you start reading, please Join Our Subreddit so you don't miss any updates. Researchers from Carnegie Mellon University recently published a paper that compares existing code models -- Codex, GPT-J, GPT-Neo, GPT-NeoX, and CodeParrot -- across programming languages. By comparing and contrasting various models, they want to offer more light on the landscape of code modeling design decisions, as well as fill in a major gap: no big open-source language model has been trained purely on code from several programming languages.